Systems and methods involving search enhancement features associated with media modules

ABSTRACT

Provided herein are mixed-media modules with enhanced features that can be used as search results. Systems and methods are disclosed for performing processing involved with search, such as processing search information to return search results. In one exemplary implementation, there is provided a method for processing information to return search results including mixed-media media presentation(s) selectable by a user. Moreover, such method may involve user interaction to manipulate the presentation, display various media and/or effect other functionality. Further implementations may involve generation of interactive, visually rich mixed-media content of high information density providing improved user experience and/or improved value to various participants.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims benefit/priority of provisional application No.61/658,909, filed Jun. 12, 2012 and entitled “Systems and MethodsInvolving Features of Search and/or Search Integration;” and U.S.non-provisional application Ser. No. 13/717,662 , filed Dec. 17, 2012and also entitled “Systems and Methods Involving Features of Searchand/or Search Integration;” each of which is incorporated herein byreference in its entirety.

BACKGROUND

Field

Aspects of the present innovations relate to computer networkingsearches, and, more particularly, to associated systems and methods,such as processing search information, providing interactive searchresults, and search integration. In particular, provided herein aremixed-media modules having search enhancement features and systems andmethods for creating the same. Mixed-media modules have more compactdata structure, more diverse data types, enhanced accuracy and canpromote search efficiency.

Description of Related Information

The web has evolved into a rich, multi-media experience, but the processof searching online and associated drawbacks have changed little in thelast fifteen years. Search is still primarily text based (captions) withonly small thumbnail images (or previews) appearing as a visual searchresult. Text captions are machine generated and are not a rich orefficient user experience. Also, humans process visual information muchfaster than we process text, but there is limited visual information insearch results. Search engines have tried to remedy this problem byproviding “live previews” of the source web pages and presenting them intext and graphical form. Unfortunately, this process is expensive,storage heavy and adds little value for the end user. Further, Internetsearch results often result in lists of hyperlinks that are not veryinformative to the searching user.

For example, FIGS. 1 and 2 show exemplary screenshots of prior artsearch result pages. These prior art examples show how generally, whenan end user performs an Internet search, the search engine produces asearch results page (also called an “SERP”). The prior art, as shown inFIGS. 1 and 2, contain lists of results with hyperlinks and a sentenceor two about each result, 101, and 201. That text, 101, 102, ismachine-selected by proprietary algorithms unique to each searchengine—as opposed to being curated by humans—and is sometimes a randomand not adequate description of the linked page. As such, there is noend-user control of the displayed text.

The selected text is called a “caption” as shown in FIGS. 1 at 101, andFIG. 2 at 201. Captions were first used when there was no rich media onthe web and, therefore, were only text-based. Because of this legacy,architecture search results are mostly text-based captions as shown inFIGS. 1 and 2, the way users consume this media is in a limitedformat—meaning that they can only view search results as one form ofmedia at any given time, such as limited to just video, or just text.

Continuing with FIGS. 1 and 2, the prior art presented results as text,still images or video. There is not a great deal of context to thecaptions in search results and the presentation of those results isdifferent from every search engine even though each search engine hasits own proprietary search algorithms. In order to refine a search inthe prior art systems, one must start a search over or hit the “back”button to return to earlier results. Further, searches from mobiledevices only compound problems in the prior art. With limited screenreal estate, proprietary operating systems, limited bandwidth and avariety of interfaces, such as touch, voice, keyboards—both on screenand physical.

FIGS. 3 and 4 are illustrations of exemplary prior art web pagepreviews. FIGS. 3 and 4 show that even when an entire page is presentedas a live preview, 301, 401—as it is with example company SERP, there isnot much value added to the user's search. The information is denselypacked and the graphics are too small to be useful. Only the generallayout of the page is discernible which does little in terms of addingcontent or context.

Another problem is that search engine results are often inaccurate andimperfect. Text captions do not always accurately represent the contenton a site because they lack context and richness. As a result, a searchmay not be efficient. Users often waste time uncovering the actualcontext of individual search results.

Currently, companies or website publishers do not have control over howtheir caption(s) appear within a SERP. The captions are algorithmicallymachine generated and cannot be curated by the owner of a site.

In sum, there is a need for systems and methods that address the abovedrawbacks an/or provide other beneficial functionality or advantages toparties involved with search.

SUMMARY

Systems and methods consistent with the present innovations are directedto implementations such as processing search information, providinginteractive search results, and search integration, among others.According to some implementations, system and methods herein may allowfor search results of improved nature, such as results that areinteractive, expanded, deeper and/or richer as a function of mixed-mediacomponents, as well as improved value to all participants and improveduser experience, among other benefits.

It is to be understood that both the foregoing general description andthe following detailed description are exemplary and explanatory onlyand are not restrictive of the invention, as described. Further featuresand/or variations may be provided in addition to those set forth herein.For example, the present invention may be directed to variouscombinations and subcombinations of the disclosed features and/orcombinations and subcombinations of several further features disclosedbelow in the detailed description.

DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which constitute a part of thisspecification, illustrate various implementations and features of thepresent inventions and, together with the description, explain aspectsof innovations herein. In the drawings:

FIG. 1A is an exemplary screenshot of a prior art search result page.

FIG. 1B is block diagram of FIG. 1A.

FIG. 2A is an exemplary screenshot of a prior art search result page.

FIG. 2B is block diagram of FIG. 2A.

FIG. 3A is an illustration of exemplary prior art web page previews.

FIG. 3B is block diagram of FIG. 3A.

FIG. 4A is an illustration of exemplary prior art web page previews.

FIG. 4B is block diagram of FIG. 4A.

FIG. 5A is an illustration of a search engine results page withintegration features consistent with certain aspects of the innovationsherein.

FIG. 5B is block diagram of FIG. 5A consistent with certain aspects ofthe innovations herein.

FIG. 6A is an illustration a live preview showing an example searchengine results page consistent with certain aspects of the innovationsherein.

FIG. 6B is block diagram of FIG. 6A consistent with certain aspects ofthe innovations herein.

FIG. 7A is a diagram illustrating an example search engine results pagefrom a re-query consistent with certain aspects related to theinnovations herein.

FIG. 7B is block diagram of FIG. 7A consistent with certain aspects ofthe innovations herein.

FIG. 8A is an example showing ad placement in an implementationconsistent with certain aspects related to the innovations herein.

FIG. 8B is block diagram of FIG. 8A consistent with certain aspects ofthe innovations herein.

FIG. 8C is block diagram depicting an exemplary process for creating anAd layer to the mixed-media module.

FIG. 8D is block diagram depicting an exemplary process for providingmixed-media module with targeted content.

FIG. 9 is an exemplary screenshot showing an illustrative mobile devicedisplay including a search engine results page with integratedmixed-media component consistent with certain aspects related to theinnovations herein.

FIG. 10 is an illustration of an exemplary search engine results pageshowing user action with a mobile device display search results pageconsistent with certain aspects related to the innovations herein.

FIG. 11 is an exemplary screenshot illustrating further mobile devicedisplay functionality consistent with certain aspects related to theinnovations herein.

FIG. 12 is an exemplary screenshot illustrating mobile device display ofa search results content such as a mixed-media module consistent withcertain aspects related to the innovations herein.

FIG. 13 is an exemplary screenshot of an illustrative mobile devicedisplay showing user interaction with a mixed-media module from thesearch results consistent with certain aspects related to theinnovations herein.

FIG. 14 is an exemplary screenshot of a mobile device display showing anillustrative result of a user interaction consistent with certainaspects related to the innovations herein.

FIG. 15A is a flow chart showing an exemplary method associated withillustrative search/mixed-media module processing consistent withcertain aspects related to the innovations herein.

FIG. 15B is a flow chart showing an exemplary method associated withillustrative search/mixed-media module processing consistent withcertain aspects related to the innovations herein.

FIG. 16A is a flow chart showing an exemplary method associated withillustrative search/mixed-media module processing consistent withcertain aspects related to the innovations herein.

FIG. 16B is a flow chart showing an exemplary method associated withillustrative search/mixed-media module processing consistent withcertain aspects related to the innovations herein.

FIG. 17A is a flow chart illustrating exemplary steps associated withperforming parsing and image selection consistent with certain aspectsrelated to the innovations herein.

FIG. 17B is a flow chart illustrating exemplary steps associated with anexemplary embodiment of the autofill functionality consistent withcertain aspects related to the innovations herein.

FIG. 18 is an illustration showing an example gesture consistent withcertain aspects related to the innovations herein.

FIG. 19 is an illustration showing an example gesture consistent withcertain aspects related to the innovations herein.

FIG. 20 is an illustration showing an example gesture consistent withcertain aspects related to the innovations herein.

FIG. 21 is an illustration showing an example gesture consistent withcertain aspects related to the innovations herein.

FIG. 22 is an illustration of an exemplary search engine results pageshowing integration/position aspects consistent with certain aspectsrelated to the innovations herein.

DETAILED DESCRIPTION OF ILLUSTRATIVE IMPLEMENTATIONS

Reference will now be made in detail to the invention, examples of whichare illustrated in the accompanying drawings. The implementations setforth in the following description do not represent all implementationsconsistent with the claimed invention. Instead, they are merely someexamples consistent with certain aspects related to the invention.Wherever possible, the same reference numbers will be used throughoutthe drawings to refer to the same or like parts.

According to some implementations, systems and methods consistent withthe innovations herein are directed to providing search results withimproved features. For example, aspects herein may relate to innovativeintegration of a rich, mixed-media, interactive component, sometimesreferred to as a ‘Qwiki’™ component or module, into search resultspages. In some implementations, this component or module may be aninteractive narrative presentation of the content that is being searchedand it may feature an interactive layer which allows the recipient ofthe search result to receive more detailed information without leavingthe search engine results page (“SERP”). According to certainembodiments, systems and methods involving search results integratedwith these component(s) may include features that are innovative overexisting systems due to the information density andmixed-media/multimedia capabilities of such “mixed-media” integratedcomponent(s). The terms “mixed-media module” and “multimedia module) areused interchangeably unless otherwise specified.

As set forth herein, implementations may involve the integration of suchcomponent into a search engine results page (SERP). This can be anyexisting or future SERP including those popular today. Moreover, variousSERP-component integrated systems and methods herein provide display ofsearch engine results in an interactive playable format compatible withmobile devices and their variety of interfaces.

In one aspect, provided herein are mixed-media modules that can bepresented to users as internet search results, which provide numerousenhanced features over conventional search results (e.g., more diversetype of media, more compact/accurate information with reduced or noredundancy, etc.). FIGS. 5A and 5B are illustrations of an exemplarysearch engine results page including an integrated mixed-media module500 consistent with aspects of the innovations herein. Suchimplementations allow the user to stay on the search page andefficiently interact with the search engine in a way that is beneficialfor that search engine through deeper more refined searches, increasedad views and clickthrough rates (CTR). Further, in various embodimentsset forth herein, the integrated component may include features thatserve as an “interactive summary” of a web page/search result whichenhances the utility of the search experience. This results in higherquality searches and the increased revenue for the search provider, suchas through re-queries (deeper searches in the existing topic).

In one illustrative implementation, for example, there is provided amethod of processing search information comprising, processinginformation to return, to a user, search results via a search engine, ina results page. The search results page, in one example includes atleast one pre-prepared, non-rendered narrative multimedia presentation.The example method further comprising, providing, for display via asearch results page, at least one interactive multimedia presentationselectable by the user. In particular, such multimedia presentation maybe a mixed-media module (also referred to as “MMM” herein) as specifiedherein. Additionally, the example method further comprising providing,as a function of a first interaction of the user with a selectedmultimedia presentation, user access to at least one of third partyinformation, web sites, content, applications and/or other multimedia.Also, the example method could include providing, as a function of asecond interaction of the user with the selected multimediapresentation, functionality configured to receive a new search query andgenerate a new search results page.

Further integrations of such components with search results also involvecreation of new ad units (and thereby, in some implementations, newsources of revenue) inside of the mixed-media interactive summary, asexplained further below.

Consistent with certain implementations, another way systems and methodsherein may depart from the traditional search experience, and fromonline video, is that the end user does not have to experience themixed-media module in a linear fashion. The user can choose their paththrough the content with various functionality, such as clicking onhyperlinks within the mixed-media module, via navigationfunctionality/gestures, and the like. This allows the end-user toexplore the information that is of the most interest to them, in greaterdetail and in their preferred format, e.g., text, photos, video, etc.

Turning again to FIGS. 5A and 5B, a mixed-media module 501 can be acontrollable media component within a search results page. Here, forexample, a mixed-media module may give the publisher of a site controlover its brand and its content as it appears on the search results pagewithin such mixed-media module. This may be accomplished via creatortools associated with creation of such modules that generate anembeddable interactive object, or via markup language that publisherscan include on their site that is recognized by search engine crawlers.This also leads to a better search experience for the end user.Implementations include the integration of a multimedia component suchas a mixed-media module into the SERP of an Internet search engine asillustrated in FIGS. 5A-5B. Such component/module's interactive summarycreates a playable caption that surfaces the best contents from thepage. The title in this illustration, for example, “Watch the Qwiki[module], Tokyo” may specified by a mixed-media module elements 501 and505.

In some embodiments, a mixed-media module is offered as a search resultin addition to a listing of traditional text-based search result; e.g.,illustrated as element 500 in FIGS. 5A and 5B. For example, one or moremixed-media modules are listed under a listing for a news articleconcerning an event or a person (e.g., the mixed-media modules arearranged in an array, a grid, or as pages of a storybook). In suchembodiments, the mixed-media modules are associated with the newsarticle and provide additional and multi-dimensional media to allow auser to explore the event or person at great depth.

In some embodiments, a mixed-media module is offered as a search resultwithout a listing of traditional text-based search result. In suchembodiments, multiple mixed-media modules are presented in an array, agrid, or arranged as pages of a storybook. In some embodiments, amixed-media module can be played by clicking on an icon representing themixed-media module; for example, the icon is marked with a play symbol.In some embodiments, a mixed-media module can be played by moving themouse over the icon representing the mixed-media module. In someembodiments, the mixed-media module search results are depicted as atoolbar (e.g., element 501 in FIGS. 5A and 5B. In some embodiments, thetoolbar includes a title 505 suggesting the content of the mixed-mediamodule. In some embodiments, the toolbar includes a play icon thatallows user to initiate or start playing the mixed-media module (e.g.,by mouse-over, clicking the play icon, a tap motion or other gesture ona mobile device). In some embodiments, the toolbar includes afunctionality for controlling the volume of the mixed-media module. Insome embodiments, the toolbar includes a functionality for controllingthe volume of the mixed-media module. In some embodiments, the toolbarincludes a functionality for adjusting the size of the visual display ofthe mixed-media module.

In some embodiments, one or more mixed-media modules are offered assearch results dynamically as a user enters a search term. For example,as a user enters a search term, a list, array or grid of mixed-mediamodules appear on a dropdown menu. In some embodiments, the mixed-mediamodules included on a dropdown menu change dynamically as the userenters and completes the search term. For example, when a user enters“t,” mixed-media modules concerning target (the store), twitter (thecompany), tea (the drink), ted (the talk) and Ted (the airline) can bepresented as search results. When a user enters “ti,” mixed-mediamodules concerning New York Times (the newspaper), ticketmaster (theticket vendor company), and Titanium (the element) can be presented assearch results. Similarly, when a user enters “tiger,” mixed-mediamodules concerning tigers (the animal), Tiger Wood (the golfer), tigerprawn (the big shrimp) and TigerDirect (the website selling electronicsand more) can be presented as search results. In such embodiments, auser can be connected with the most relevant result bycompleting/providing more details to the search terms or simplyselecting the most relevant result among the dynamically providedmixed-media modules. Exemplary method steps concerning the autofillfunctionality is depicted in FIG. 17B.

In one aspect, mixed-media modules provided as internet search resultsthey present diverse, more in depth, and multi-dimensional informationthat easily encompass content found in multiple, dozens or even hundredsof conventional search results (e.g., FIGS. 1-4). In some embodiments, amixed-media module serves as a platform for presenting more diverse,compact, multi-dimensioned information. For example, the mixed-mediamodule depicted in FIGS. 6A and 6B include combines graphic (e.g., aninteractive map), audio (e.g., music or audio caption), image (picturesrelevant to the search term), video and textual information in anintegrated form. Using a toolbar similar to one described in connectionwith FIGS. 5A and 5B, a user can navigate the mixed-media module andselect the viewing experience by choosing the desirable audio and videosettings, as described in details hereinbelow.

FIGS. 6A and 6B depict exemplary preview illustrations showingillustrative search engine results pages with mixed-media modules.Consistent with this basic preview as explained herein, a mixed-mediamodule integrated into the search results page provides for a richeruser experience and increases traffic for that page. Further,implementations may include playable captions that provide more contextthan regular text captions used in existing systems.

Consistent with the innovations herein, systems and methods are providedinvolving procedures and/or mechanisms for enhancing search results vianovel integration of mixed-media modules. Such implementations maypresent coordinated text, images, video, documents, narrations and linksall available in one interactive screen format or window. Examples ofthese can be seen in FIGS. 6A and 6B. Here, for example, the mixed-mediamodule may be a rich multimedia visual and interactive piece of content.A search results page, SERP, integrated with such mixed-media moduleacts as an interactive multimedia summary of a search result rather thanjust a text based caption integrated into an SERP as previously done.

As seen in connection with FIGS. 6A and 6B, the typical search engineresult is augmented or even replaced by a mixed-media module 601 thatenhances the results. Navigation to a desired result, e.g. a selectedmixed-media module, may be an expansion inside the normal search resultsinto a larger display. Further, a ‘new search’ button, icon orfunctionality may be included within mixed-media modules, e.g., amagnifying glass icon 603. This may be configured to allow for furthersearching or re-querying within the mixed-media module. Further, amedia/asset loading bar 605 may also be included, allowing for audioand/or video to play in the mixed-media module or in another window. Themixed-media module may also include one or more hyperlinks 610 to otherweb pages. An expander button 615 may also be included to allow for themixed-media information in the module to be displayed in a full screenformat.

With regard to these implementations, such as ‘new search’functionality, systems and methods herein may involve methods ofprocessing search information comprising a computer server configured tocommunicate with at least one search engine web crawler. Exemplarymethods also may include the computer server configured to receive thesearch engine web crawler results from at least a first query, and togenerate search results for display in a browser window based on thefirst query. Methods may also include embodiments involving provision ofsearch results include at least a customizable caption, variousmultimedia content, and at least one hyperlink configured to cause are-query of the search engine web crawler.

Referring still to FIGS. 6A and 6B, in accordance with some aspects ofthe innovations herein, a mixed-media module integrated SERP alsoimproves the usefulness of search. A seen in FIGS. 6A and 6B, suchinteractive component has a higher density of information than the priorart, which proves to be more valuable to the end user, online contentproviders, and the search engines. The search engine crawlers can detectcertain mixed-media module, such as via detection of metadata associatedwith Qwiki™ modules, and embed it in a search results page (SERF).Further, implementations herein may utilize the mixed-media module as aninteractive and playable caption, 605.

According to implementations herein, once played, the mixed-media modulemay expand within the page as shown in FIG. 6 at 601 and can offer theuser a variety of options to explore related content triggering newsearch queries, 603, media/asset loading, 605, links to related pages,610, and playback options, 615. Further, component video or audio filesmay be played within the mixed-media module on the SERP, without needfor loading an external page.

In some embodiments, a user can navigate or view the content of amixed-media module in a non-linear fashion; for example, a user canexplore the interactive map depicted in FIGS. 6A and 6B. Alternativelyor in combination with the interactive content, a user can select anembedded executable link (e.g., element 610) to go to a relevantexternal website. In some embodiments, the external website providesadditional information about a specific piece of content of themixed-media module. In some embodiments, the external website offers theuser the choice to browse possible commercial websites that are relevantto the mixed-media module. For example, links to hotels and restaurantsin Tokyo. In some embodiments, the embedded links are opened as new tabswhile the mixed-media module is preserved in the original web interface.In some embodiments, the embedded links are opened as new windows whilethe mixed-media module is preserved in the original web interface. Insome embodiments, the embedded links are opened as new split-screenwindows in addition to the interface in which the mixed-media module ispresented. In some embodiments, a user can select a specificsplit-screen window by a mouse-over motion, a clicking motion, or atapping motion (e.g., on a mobile device or a touch screen device).

In addition to the display of related media/links in a new window on thesame page, further implementations provide functionality to displayassociated content on the same SERP by instantly modifying it or thecontents around it. Here, for example, this new integrated content maybe displayed without triggering new tabs.

Additionally, in contrast to existing SERP functionality where captionsare algorithmically machine generated and cannot be curated by relevantparties, systems and methods herein may provide a controllableinteractive media component within a search results page. For the firsttime, then, implementations herein involving the mixed-media modulesallow the publisher of a site control over its brand and its content asit appears on the search results page within the mixed-media module.

Further, consistent with certain aspects related to the innovationsherein, present implementations improve upon and enhance existing searchtechnologies because they provides narrative context to searchresults—something lacking until now. Results herein are a richerexperience with more visual, linked information and interactivefeatures.

As a function of the present mixed-media modules embodiments, which maybe created by participants of the search process, search results may bemore accurate and provide better context. Consistent withimplementations herein, brand managers and content publishers cancontrol their story within a search engine result without purchasingexpensive search advertising. This is particularly valuable becauseexisting captions are often not relevant for a search engine user theyadd little or no value to the process. They contain a limited amount ofdata and few clues as to the overall content of the site those captionsare supposed to summarize. In other words, captions lack the context andvisual richness provided via the innovations herein. Additionally,search engine results are clustered in a way that isn't helpful and canbe overwhelming. Users get results that don't help with a decisionbecause they are unrelated to what the user actually needs. The limitedtext in a caption often doesn't reveal enough information. As a resultthe user must select links, search that site and, if it is not thedesired result, back up to the original search results or begin a newsearch from scratch. It's time consuming, awkward and makes things easyfor a user to get lost.

In some embodiments, accuracy of search results is improved whenredundant results are simplified or consolidated when a mixed-mediamodule is created. For example, instead of dozens or hundreds of imagesof Tokyo, a mixed-media module only includes a few most representativeimages. In some embodiments, redundancy is detected by accessing themeta-data associated with the content (e.g., an image, a video or anexcerpt from an article) that can be possibly used in a mixed-mediamodule (e.g., only one of the most referred to image will be included).For example, content originated from the same source will be used onlyonce. In some embodiments, meta-data associated with a piece of content(e.g., an image, a video or an excerpt from an article) can also be usedfor prioritizing the content; for example, the popular image or videowill be highlighted in the mixed-media module (e.g., displayed in alarger size or over a longer period of time). As a consequence andadvantage of the more accurate smart content selection, a more compactand efficient mixed-media module can be created. In addition, more typesof information can be included in a mixed-media module becauseredundancy is removed or reduced.

Advantageously, a mixed-media module avoids respective information.Consequently, mixed-media modules provided herein present informationmore efficiently (e.g., by combining information that are available frommultiple conventional search results), more accurately (e.g., themulti-dimensional data presented would help a user to make selectionmore accurately; redundant information is removed or occurred lessfrequently when a mixed-media module is created). Also advantageously,the multi-dimensional and in-depth presentation of diverse information,which is both rich in content and format, tends to increase the dwelltime of a user, which is important for an information provider ordistributor.

According to further embodiments, a search result enhanced via presentmixed-media module(s) implementations may also involve innovationsassociated with second or follow-up queries, referred to herein as“re-query.” FIG. 7 is a diagram illustrating an example search engineresults page associated with a re-query, consistent with aspects of theinnovations herein. Notably, a re-query allows a search engine user torefine their search results without losing the original search. Clickingon a hyperlink within the mixed-media module allows the user to“re-query” the search engine and dig deeper into a subject by searchingthe mixed-media/interactive components within a module. Implementationsare also configured such that this opens a new window without closingthe original one and thereby reduce the need to constantly hit the“back” button in order to return to the original results. This enablesthe ability to search, and then re-search specific details of interestwithin a search result without getting distracted or lost.

In one illustrative implementation, for example, there is provided amethod of processing search information comprising processinginformation to return, to a user, search results via a search engine, ina results page. This example method could also include where the resultspage includes at least one pre-prepared, non-rendered narrativemultimedia presentation, such as a mixed-media module. Further, theexample method could include providing, for display via a search resultspage, at least one interactive multimedia presentation selectable by theuser. Also, providing, as a function of a first interaction of the userwith a selected multimedia presentation, user access to at least one ofthird party information, web sites, content, applications and/or othermultimedia. And the example method could also include providing, as afunction of a second interaction of the user with the selectedmultimedia presentation, functionality configured to receive a newsearch query and generate a new search results page.

Various “re-query” implementations also allows users to stay on a searchpage and refine their searches in new windows without losing theoriginal search or getting lost. This is more efficient for users andless frustrating as they are more likely to find their desired results.Systems and methods herein may be configured to refine a SERP via suchfunctionality, allowing for high information density. For example, there-query can show selected caption with images 705. It can also showvideo or animation 710. Moreover, specific concepts may even besuggested for further re-query 715. In addition to the display ofrelated media/links in a new window on the same page there is an optionto display associated content on the same SERP by instantly modifying itor the contents around it. In some embodiments, this new integratedcontent is displayed without triggering new tabs. In some embodiments,this new integrated content is displayed in new tabs. In someembodiments, the new integrated content is presented as new split-screenwindows in addition to the interface in which the mixed-media module ispresented. In some embodiments, a user can select a specificsplit-screen window by a mouse-over motion, a clicking motion, or atapping motion (e.g., on a mobile device or a touch screen device).

In some embodiments, a “re-query” implementation allows a user to searchand identify other mixed-media modules as results of a new query, thusadding another level of multi-layered content to the search results. Forexample, referring to FIGS. 7A and 7B, a user can search “Tokyo ImperialPalace” within a general mixed-media module result concerning thebroader search term “Tokyo.” In some embodiments, in response to the newquery, conventional search results (e.g., as illustrated in FIGS. 1-4)are presented. A user can access the conventional search results inmanners similar to the access to embedded links. In some embodiments, inresponse to the new query, new and more specified mixed-media moduleresults (e.g., mixed-media modules with more detailed and in-depthcollection of information concerning “Tokyo Imperial Palace”) arepresented. A user can access the mixed-media module results in mannerssimilar to the access to embedded links; e.g., as new tabs on a browseras split-screen windows. A user can navigate between the originalmixed-media module and one or more new mixed-media modules by amouse-over motion, a clicking motion or a tapping motion (e.g., on amobile device or a touch screen device).

These “re-query” innovations may also drive a deeper understanding of aqueried subject matter by displaying related search topics. As such,systems and methods herein provide for a mixed media/multi-mediacapability which can illustrate/enhance a selected search result withimages, videos, animations, documents and even narrations. Specificconcepts can be suggested for re-query, driving additional search enginetraffic. This additional traffic yields higher advertising rates on there-query pages as the searches are more specific and focused by a morespecific customer interest. The richness of the re-queried media alsoachieves beneficial advertising results, given that richer media fetchesan increased CPM/CPT (Cost Per Thousand impressions) that advertisersare willing to pay.

Systems and methods herein overcome other issues with search engineresults, namely problems associated with search placement. Placement onthe SERP is important because, the higher the placement, the more likelya site will be selected by a user. For this reason, the top of the pageis seen as the most valuable real estate on an SERP. Entire industrieshave been created just to place a search result in a higher position inthe SERP as processing pages and pages of text results is timeconsuming.

Presently search engines consider it a success when a user spends aminimal amount of time on their page. This might be counterintuitive,but it's because a quick search process means that the user is findingthe information that they need and moving on. The down side to this isthat the search engine only has a limited amount of time to display adsand monetize the interaction. As such, implementations herein provide aninnovative and ideal scenarios for search engines, e.g., keeping userson their site through a layer of interactivity that allows for a deeperexploration of search results without leaving the original search timeand time again.

While visual results in searches can yield better results, previews ofwebsites are very expensive for search engines to create, maintain andstore. Bandwidth is also an issue when end users access search enginesvia mobile devices. On mobile devices and smart phones, in particular,there is limited screen real estate and text-based search results aretiny and difficult to read. It's even more difficult for a user todifferentiate between search results when looking at a tiny screen.

Moreover, many search engines are adding video content to their searchresults. Video is becoming more prevalent online because publishersdon't want to present text-only sites and there is a desire todifferentiate/supplement search placement; however, traditionalstreaming video is time-consuming to create and view. Video content isalso highly compressed on mobile devices resulting in poor streaming andpicture quality. Video is also hard to interact with because there is nostandard, universal layer for interactivity. For the most part, video isa stand-alone experience because of the lack of an interactive layer. Inaddition, similar to exploring component web pages, watching andre-searching for appropriate videos is very time consuming—because oflimited previews, users often don't know if they have discovered theright or wrong video related to their topic, as the videos are indexedand retrieved via keyword, not according to the content of the pagesalso part of the same search result.

Embodiments herein address these issues and drawbacks, as well. In oneillustrative implementation, for example, there is provided a method ofprocessing search information comprising a computer server configured tocommunicate with at least one search engine web crawler. The examplemethod could also have the computer server configured to interact withthe search engine web crawler search results by causing display of thesearch results. And the example method may include wherein the searchresults include interactive multimedia content, e.g., one or moremixed-media modules, and/or associated content such as at least onehyperlink, etc.

Especially in view of the issued with traditional video content notedabove, systems and methods herein are an improvement on other rich mediasuch as online video technology because they use less bandwidth, areeasily customizable, flexible, incorporate interactive video, images,text and other types of media.

In still other exemplary embodiments herein, mixed-media moduleintegrated implementations can incorporate interactive images, text andother types of media. Further, given such implementations operatewithout large-bandwidth video transmissions especially rendered videocontent for audiovisual/multimedia experience, systems and methodsherein provide an expanded interactive search with other mixed media,thus allowing for quicker loads and consumption of less bandwidth duringutilization.

Advantageously, the richness of a search result is further enhanced bythe query-within-a-query result platform within a mixed-media module.Advantageously, search efficiency and accuracy are also enhanced by thequery-within-a-query result platform.

In one aspect, a mixed-media module provided herein serves as a gatewayto additional information (e.g., information of vendors or stakeholderssuch as commercial web sites for the vendors or stakeholders). Anyvendors or stakeholders, directly or indirectly associated with thecontent of a mixed-media module, can be used in embodiments describedherein. For example, a mixed-media module of “Tokyo,” can includeinformation of hotels, stores, museums, travel agencies, airlines,places of interests (e.g., hot springs), hospital.

FIG. 8A is an example showing illustrative ad placement 801 features,consistent with aspects of the innovations herein. The integration ofmixed-media module interactive summaries into a SERP creates additionaladvertising monetization units, these units can be presented asinteractive captions on the CPC/PPC (Cost Per Click/Pay Per Click)advertisements that traditionally are placed alongside search results,or the CPC/PPC ads (and other promotional units) can be placed withinthe mixed-media module itself, as shown in FIG. 8. For example, theinteractive summary can be presented as a caption on the CPCadvertisement that are traditionally placed alongside organic searchresults 801. In some implementations, the CPC ads can be placed withinthe multimedia presentation or mixed-media module 805, itself.

It should be noted that FIG. 8 may give the appearance that the CPC adis loading within the Wikipedia result. However, implementations mayinclude the CPC ad displaying its own mixed-media module. Loading theCPC ad into the Wikipedia mixed-media module is a different embodimentfrom such implementations.

As described hereinabove, a mixed-media module presents information withremoved or reduced redundancy. Therefore, it is possible to include morediverse information in a mixed-media module, which makes advertising viaa mixed-media module more efficient. For example, a mixed-media moduleconcerning “Tokyo” may include advertising information of differenthotels (e.g., FIGS. 8A and 8B) where a user can browse a list of hotels(e.g., elements 801) and select one or more to view in detail (e.g.,element 805). The same mixed-media module concerning “Tokyo” can includeadditional and relevant advertising information of restaurants, stores,museums, and places of interests (e.g., hot springs). As such, it ispossible to include more diverse, more detailed advertising informationfrom numerous types of vendors or stakeholders. In addition, relevantinformation although not directly associated with the content of amixed-media module, such as sites for purchasing travel tickets can alsobe presented in a mixed-media module concerning “Tokyo.”

In some embodiments, information of the vendors and/or stakeholders isdetermined based on the preference or profile or a particular user. Forexample, a user can permit a mixed-media module content provider toaccess a personalized profile of the user. In some embodiments, one ormore mixed-media modules are generated dynamically as results, forexample, on a local device or a remote server, in response to a searchterm. In some embodiments, one or more mixed-media modules areidentified among a library of pre-existing mixed-media modules on aremote server. In some embodiments, a combination of one or moredynamically generated mixed-media modules and one or more identifiedpre-existing mixed-media modules are provided as results in response ofthe search term entered.

In some embodiments, a functionality is used to identify whether apersonalized profile exists for the user entering the search term. Inembodiments where such a profile does not exist, search results (e.g.,one or more dynamically generated mixed-media modules, one or moreidentified pre-existing mixed-media modules, and/or a combinationthereof) are provided to the user. In embodiments where such a profileexists, search results (e.g., one or more dynamically generatedmixed-media modules, one or more identified pre-existing mixed-mediamodules, and/or a combination thereof) are modified before beingprovided to the user. In such embodiments, information of vendors and/orother stakeholders is dynamically added to relevant content in amixed-media module in the search results, which include, e.g., one ormore dynamically generated mixed-media modules, one or more identifiedpre-existing mixed-media modules, and/or a combination thereof. In someembodiments, multiple pieces of information of vendors and/or otherstakeholders are dynamically added to relevant content in a mixed-mediamodule in the search results.

An exemplary process for adding an Ad layer to an existing mixed-mediamodule is depicted in FIG. 8C. In some embodiments, mixed-media moduleswith different types of advertisement information and/or otherstakeholder information are pre-created and categorized in one or moredatabase on a remote server. In some embodiments, mixed-media moduleswith different types of advertisement information and/or otherstakeholder information are created on-the-fly based on recent or realtime activities of a user. For example, hotel information will beprovided if the user has previously search hotels before submitting thequery “Tokyo.”

A personalized profile is loosely defined; it can be created/identifiedbased any information/preference/interests/tendency/trend in activitiesthat are associated with a user. In some embodiments, the personalizedprofile includes search preferences of the user; e.g., created bytracking the search activities of the user; by providing the means/userinterface for the user to enter such information; by tracking the user'sinterest list on a website such as Yahoo!, Facebook, Google, Twitter, orTumblr; or by importing an existing preference profile of the user fromon a website such as Yahoo!, Facebook, Google, Twitter, or Tumblr. Insome embodiments, the personalized profile includes one or more searchresults from the user. In some embodiments, the personalized profileincludes one or more people, one or more topics, or one or more events,associated with the user; for example, the list of people the userfollows on Twitter.

In some embodiments, a personalized profile can be created by providingan interface for the user to select or enter information concerning theuser's preference or interests. For example, a fillable form or aninterface where listings of interests/preferences are provided and auser can select particular interest/preference by check a correspondingbox.

In some embodiments, a personalized profile is used to suggest content,including possible mixed-media modules that are of interest to theparticular user. This capacity is particular important to contentprovider and distributor. For example, a personalized profile includesthe user's interests/preference in news or other types of content. Morespecifically, a user can have a preference for technology news (e.g.,following one or more types of technologies; one or more types ofindustries; and/or one or more companies), political news, or news fromone or more selected geographical regions. In some embodiments, one ormore mixed-media modules are provided to a user, not in response to asearch term, but as part of the preferred content on a portal for theuser; for example, a personalized Yahoo! Page such as My Yahoo! In suchembodiments, the user logs into the portal page via an authenticationmodule and a personalized profile can be created for the user. Forexample, the user can be provided with an interface through which toselect specific topics of interest that will be included and saved inthe personalized profile. The content provider/distributor thenprovides/distributes specific content targeting the user based on thepersonalized profile.

A mixed-media module as an advertising tool offers many advantages. Forexample, a mixed-media module offers more diverse and more interestingcontent compiled using multiple media formats. Also, a mixed-mediamodule has enhanced and superior advertising capacity—with the SERP of amixed-media module, each component in the layout can be used as aplatform for advertisements—thus making it possible to promote multipletypes of vendors and/or stake holders at the same time. In someembodiments, different positions are auctioned or offered to vendors andother stakeholders. For example, a top Italian restaurant may pay toobtain a prominent ad spot (e.g., near the beginning of a mixed-mediamodule) among mixed-media module search results that are provided inresponse to a “San Francisco Italian restaurant” query.

In another illustrative embodiment, as illustrated in FIG. 8D, one ormore mixed-media modules are created uniquely for each user. The processmay begin at step 825 where a user navigates to a web site, such as aweb portal, including a login interface where the user enters theircredentials and is authenticated. Once logged in, at step 830, access toa personalized profile is made. The user is identified by the systemwith a corresponding personalized profile retrieved from a profiledatabase that includes any number of user defined settings. For example,the user profile may include user input data regarding their name,hobbies, likes, dislikes, web portal preferences, contact information,friends, purchases, application information, language, posts, tweets,and the like. Other user profile information may include social networkconnections such as the people a user follows on Twitter or on othersocial media networks. The user profile may also include informationcollected while the user is logged into the web portal, but nottransparent to the user, such as information related to their usage onweb sites, user search history and so forth. Together, this user profileincludes data that are actively collected directly from the user, andpassively collected based on user activity.

A determination is made at step 840 of whether a personalized profile isexists for the authenticated user. If not, and it is determined at step850 that it is not possible to create a personalized profile (if a userdeclines or opts out), then at step 885, the mixed-media modules thatare selected for presentation to a user will be based on criteria otherthan the personalized profile. For example, the top ranked, most sharedor most recent mixed-media modules may be provided to a user at step885.

However, when no personalized profile exists, but it is possible tocreate a personalized profile, then a user creates a personalizedprofile at step 855. Once the personalized profile is created orretrieved, mixed-media modules are searched for and identified based onthe personalized profile at step 865. Mixed-media modules may also becreated in real-time in the case of a breaking news story, or updatedinformation based on the personalized profile. For example, a userprofile may indicate strong interest in a music group such that aplurality of databases are searched for content and mixed-media modulesrelated to the music group for presentation to the user. If a newannouncement is made by the group, a new mixed-media module isdynamically generated based on the user's declared interest for themusic group and the group's new announcement. Other users with the sameinterests may also be presented the dynamically generated mixed-mediamodule. The databases searched are not limited to those in which theuser is logged into, but may be any database include those containingpreviously created mixed-media modules, user-created mixed-mediamodules, and dynamically generated mixed-media modules.

At step 875, the identified modified mixed-media modules are provided toa user, and may even include a byline with the user since themixed-media module is formed based on the user's personalized profile.In this manner, users of different profiles are provided mixed-mediamodule content that will most likely engage each user, thereby promotinguser dwell time within a web portal.

Moreover, mixed-media modules that are created are stored and indexed ina database for use by the web portal and/or search engine crawlers. Theindexing may be based on the metadata within the mixed-media module. Themixed-media module that is richer in content for a given topic may theneasily become integrated into search engine results for the user andothers. The real-time, in-depth and information rich mixed-media moduleimproves upon user's ability to search for and process information.

In another implementation, information/news event determined to be ofhigh public value as determined by any of popularity, ranking,government notice, or other criteria are dynamically generated andpresented to users. For example, an earthquake that occurs haswidespread and immediate effects on a population, and may beautomatically generated based on updated information retrieved from theUSGS. Those users living within or near the geographic area, or usersotherwise having a connection to the area within the earthquake radiusare presented with a mixed-media module related to the earthquake. Themixed-media module may include detailed information regarding theearthquake, but may also include public service information relevant topublic safety, such as links to shelters, traffic alerts, and otherrelevant topics. In this manner, the mixed-media module provides publicnotice to important events.

Mixed-media modules may also be generated for other types of emergencyevents such as natural disasters and terrorist attacks, or other ongoingcriminal activity (i.e. Amber Alert). The users who receive thesemixed-media modules may be limited, for example, users in Alaska mayhave no need to receive a mixed-media module related to a tornado in theMidwest. However, if the user profile indicates that the user in Alaskahas family from the Midwest, then that user may receive a mixed-mediamodule related to helping the victims of the tornado, as onenon-limiting example.

Referring now to FIGS. 9-14, implementations herein with mixed-mediamodule integrations involving video can yield improved/higher quality onmobile devices, consistent with aspects of the innovations herein. Inone illustrative implementation, for example, there is provided a methodof processing search information comprising returning search results ina search results page including one or more pre-prepared narrativemultimedia presentations. The example method could also includeproviding at least one integrated multimedia presentation selected by auser. And, also, providing access to at least one of additional thirdparty information, sites, content, applications and other multimedia.Further, the example method could include wherein, the multimediapresentations are configured in association with other features forlow-bandwidth (e.g., non-rendered, etc.) display for use on a mobiledevice.

Also, given the flexible and non-rendered nature of the mixed-mediamodules, streaming and picture quality can be easily optimized forspecific mobile devices. Further, such implementations allow ease ofinteractions by providing a standard universal layer for interactivity.In other embodiments, systems and methods herein may include featuresand implementations involving interactive and coordinated hyperlinks fordeeper exploration of the content within the video—this feature ofcoordinating links/content inside of the mixed-media module interactivesummary allow new attribution and monetization capabilities by contentcreators and search engines utilizing the underlying model(s).

Here, it should be noted that a “mobile device” can be any kind ofsmartphone, tablet computer, laptop, notebook, or any kind of similardevice. These devices are typically touch screen enabled and retaininternet connectivity through either a shorter range radio such as thoseused in WiFi technologies, or through cellular telephone connections, orboth. The device may connect to the internet in any fashion.

FIG. 9 depicts an illustrative SERP with mixed-media moduleimplementation formatted for a mobile smartphone or tablet computer,consistent with aspects of the innovations herein. As shown, forexample, an illustrative “Play Qwiki module” icon is shown directlybeneath the first search result in the search result screen.

FIG. 10 is an illustration of a search engine results page with theintegration of touch-enable functionality consistent with aspects of theinnovations herein. In FIG. 10, a user is shown tapping the “Play Qwikimodule” icon using a finger. Touch enabled screens allow suchinteraction with a stylus or other such device as well, while suchfeatures may be navigated with various cursor-based functionality, aswell.

FIG. 11 is an illustration of exemplary mobile device display andfunctionality consistent with aspects of the innovations herein. In theexample in FIG. 11, the mobile smartphone may be rotated to initiate aspecified function associated with the SERP or just to allow for alandscape display, instead of a profile display.

FIG. 12 is an exemplary screenshot illustrating mobile device display ofa search results content such as a mixed-media module consistent withcertain aspects related to the innovations herein. FIG. 13 is anexemplary screenshot of an illustrative mobile device display showinguser interaction with a mixed-media module from the search resultsconsistent with certain aspects related to the innovations herein. FIG.13 shows a user interacting with a portion of the mixed-media module,here tapping the particular media or object with respect to whichadditional content (details, information, etc.) or further functionality(“re-query”, etc.) is desired. As set forth elsewhere herein, the searchengine may be configured to interoperate with such action in a varietyof ways.

FIG. 14 is an exemplary screenshot of a mobile device display showing anillustrative result of a user interaction consistent with certainaspects related to the innovations herein. Here, this example shows anillustrative re-direct associated with the tapped object to a particularweb page. The result shows a multimedia text and image or video withinthe web page.

Various aspects of the innovations involving utilization of mixed-mediamodules, as may occur in certain search contexts, are next described. Insome implementations, for example, integration of such mixed-mediamodules into search results may include features of parsing unstructureddata into structured meta data, and selecting media to be displayed.

Further, in some of the illustrative (non-limiting) examples shown anddescribed herein, search-related innovations may involve classificationfeatures and/or obtaining a common identifier in connection withmixed-media module processing, whereas other implementations may use adata source tied to the base reference or article, for example, fromWikipedia, etc., directly. To facilitate auto-generation features, someimplementations may also involve processing mixed-media modules throughan input file generator process, which may generate an input file forsubsequent processing and determinations regarding the mixed-mediamodules. Such determination(s) may include running through a classifier(with criteria) to verify suitability. Implementations may then publishor otherwise distribute mixed-media modules that satisfy the criteria.

The implementations immediately above may also include or involvefeatures of parsing unstructured data into structured meta data andmedia selection. For example, such features may include innovationsrelating to detecting entities in text such as people, places, things,etc. in text.

According to some implementations, auto-generation and near real-timemixed-media module searches and receiving structured data in general maybe realized by running a script on an information provider/server. Suchimplementations may obtain structured data, such as to run script one.g. Wiki re: updates, including obtaining the data from a setartifact/article, such as one that involves structured data. It may thengenerate the mixed-media module as a function of the structured data.Next it may publish a structured-data mixed-media module on thewebsite/venue such as a search provider, upon approval of satisfactionof criteria. The features of parsing unstructured data into structuredmeta data, and selecting media see may also be included in someimplementations. Further, these features can include innovationsrelating to detecting entities in text such as people, places, things,etc. in text.

Auto generation and near real-time Qwiki™-enable search upon death of anindividual may involve features similar to the above, which areconfigured as a function of structured data, but specific to death of anindividual.

Here, for example, implementations may first obtain a list of one ormore dead people such as to run script on an information providerserver, e.g. Wiki re: death updates, and gets this from theartifact/article, wherein structured through data: full name, age,precise moment of death, etc. Subsequent steps may includeauto-generating a mixed-media module regarding the dead person, thenpublish a death notice mixed-media module on the website/venue such as,for example, a Search provider, upon completion of an approval processof satisfaction of criteria. Again the features of parsing unstructureddata into structured meta data, and selecting the media to be shown mayalso be included in some implementations. Further, these features caninclude innovations relating to detecting entities in text, for example,people, places, things, etc in text.

Illustrative examples of some but not all implementations above areshown in FIGS. 15A-17. The following are further set forth in the codeAppendix of U.S. provisional patent application No. 61/658,909, which isincorporated herein by reference in entirety.

FIG. 15A shows an example flow chart including steps for processingscript(s) to parse unstructured data into structured data as a functionof base reference 1502 (e.g. common identifier, etc.). Next, a step ofperforming an autogeneration process to generate multimedia module(s)1504 may be performed. Autogeneration is performed by storyboarding themetadata based on a predetermined criteria, such as search enginerankings or search engine index, for example. The form, function, size,length, speed and other aspects of the presentation of the multimediamodule are variable and may be predetermined. Then, an input filegenerator processing analysis 1506 may be performed. A step ofperforming classification analysis involving scoring criteria 1508 maythen be performed. These steps add or update a search provider indexingsystem to improve further search queries. Further, performing processingon multimedia module(s) that meet scoring criteria to prepare forpublication 1510 may occur. In one implementation, the multimedia moduleis presented to another user for review prior to publication.

In one illustrative implementation, a user inputs a search query into aweb-based browser such as a search engine. The base reference, forexample, corresponds to the search query. An analysis is then performedon the search query for real-time or near real-time updated informationrelated to the search query from any number of databases. The type andnumber of databases searched is not limited to, but may includeWikipedia, news websites, social media networks, advertiser content,user-generated content, etc. For example, if the search query is relatedto a celebrity, the real-time updated information may be related to thecelebrity's most recently updated activity. If the search query isrelated to a sports team, the real-time updated information may berelated to the team's last game, in-game summary, etc. Other searchqueries may relate to a news event, such as a natural disaster, crime,etc. An example of a celebrity related search query, such as a celebritydeath, is described below with respect to FIG. 16B.

The data from the database sources is initially unstructured, but areprocessed/parsed based on their relevance to the search query and formedinto metadata. The metadata is then used to generate an interactivemultimedia module. Metadata may include, but is not limited to text,image, video, audio, document, text entry, interactive icon, etc. Themultimedia module is not necessarily interactive in someimplementations. Once generated, the multimedia module is published on aserver such as a search engine, social network, website or database. Theinteractive multimedia module may be shown as a search result within asearch engine results page in response to the input search query fromthe user. In some implementations, verification is performed of theaccuracy of the real-time updated information prior to publishing themultimedia module.

The metadata and the characteristics of the multimedia module may bebased on predetermined criteria, such as popularity and/or thosedetermined by an advertiser. The databases, unstructured data andmetadata may also be ranked based on a user profile, user searchhistory, user preferences, user customization, social networkconnections, and advertiser. For example, user search history mayindicate that a user frequently visits only a handful of websites, forwhich metadata parsed from those sites is ranked higher than metadatafrom other websites. Similarly, the databases may be selected based on auser search history.

The metadata is then storyboarded to generate the multimedia module. Themetadata may also be used to update a search engine index.

In one illustrative implementation (e.g., as depicted in FIG. 15B, steps1512 through 1520), an input of a search query related to a news eventfrom a user is input. Then, at least one database is analyzed forreal-time updated information related to the news event. Data related tothe search query is parsed from the at least one database into metadata.An interactive multimedia module is generated based on the metadata. Theinteractive multimedia module is generated on a server.

FIG. 16A shows example steps including processing structured dataanalysis operation(s) 1602. Then, performing an autogeneration processto generate multimedia module(s) 1604. Next, performing a structure datavalidation processing analysis 1606. Finally, performing transferfunction processing to prepare multimedia module(s) for transmission/useby recipient 1608.

In one illustrative implementation (e.g., as depicted in FIG. 16B, steps1610 through 1618), a user inputs a search query regarding a death of aperson or a person's name at step 1610. The person may be, for example,a celebrity. Then, at least one database is analyzed for real-timeupdated information related to the person (i.e. their death) at step1612. The real-time updated information may include biographicalinformation including a name, an age, a moment of death, and life of thedeceased at step 1614. The real-time updated information may be added tothe metadata and presented within the multimedia module at step 1616.Data related to the person is parsed from the at least one database intometadata. An interactive multimedia module is generated based on themetadata. The interactive multimedia module is generated and publishedon a server at step 1618.

FIG. 17A is a flow chart illustrating steps for performing parsing andimage selection. In some example implementations, such features may beinnovative on their own. As set forth, some implementations may involvefeatures including innovations relating to detecting entities in textsuch as people, places, things, etc. Turning back to FIG. 17A, anillustrative method may include performing processing relating to basereference (e.g. common identifier, etc.) 1702. Next, performing parsingand/or image selection functionality 1704 may be performed. Finally,performing additional processing associated with formation of themixed-media module 1706 may also be performed. In one illustrativeimplementation, duplicate metadata parsed from different databases areremoved such that any multimedia module generated will not presentduplicate content. For example, metadata parsed from the search queryterm “Paris” may produce a number of duplicate images of the EiffelTower, which would be redundant to present in a multimedia module, thusthey are removed prior to publication.

In another illustrative implementation (e.g., as depicted in FIG. 17B,steps 1710 through 1720), the multimedia module may be generated as auser is inputting each input character into a search engine text box. Amultimedia module is generated each time a new input character isentered into a text box, even before the user confirms entry by hittingan “enter” or “return” key. Based on the incomplete input character fromthe user, a search query is inferred and the multimedia module isgenerated based on the inferred search query, and therafter displayed tothe user. For example, an incomplete input character string is receivedfrom a user. Then, a search query is inferred based on the incompleteinput character string. Data related to the inferred search query isparsed from at least one database into metadata. An interactivemultimedia module is then generated based on the metadata. Theinteractive multimedia module is published on a server prior to inputcharacter string completion from the user.

Turning back to more general aspects, an illustrative multimediapresentation herein may be configured as an interactive system ofmixed-media/interactive content with clickable components. Variousmixed-media modules, here, may also provide a visual confirmation ofsearch results which means less frustration and more productivity forthe user. These mixed-media modules may also provide visualrelevancy—the multimedia nature of such interactive component providesmore in-depth detail of a topic than text alone.

Further, it is noted that pages with multi-media components are oftenranked higher in search engine results. In accordance with aspects ofthe present innovations herein, systems and methods herein provide waysfor content creators to provide interactive multi-media content and, insome implementations, improve their search engine ranking throughincreased meta-data information. The visual nature of embodiments hereinalso means that such result would not have to be ranked at the very topof an SERP to catch the attention of a search engine user since visualimages are more efficiently scanned than text. For online advertisers,better search results will mean greater return on investment. Online adswill be viewed within a more appropriate context and, therefore, morelikely to target the right consumers. Interactions with the associatedmixed-media modules can also provide additional data to rank pages.

In accordance with aspects of the present innovations, mixed-mediamodule interactive summaries as integrated herein are lightweight—theyuse less bandwidth than pure video and are a rich, interactive,multi-media experience. Viewing such mixed-media module is faster andeasier than video alone because they are interactive and have morediscrete sets of contents that can easily be traversed beyond a simpleplay bar associated with most traditional video. Mixed-media modulesherein also contain more information (meta-data) than video because ofits multitude of components (mixed media), interactive nature andbecause of the ability to re-query.

With regard to certain aspects of the innovations herein, another waythat implementations herein are an improvement over the traditionalsearch experience, especially from online video, is that that end userdoes not experience the mixed-media module in a linear fashion. A usercan readily jump to different collections of media once a quick scanassures them the preset set of options will not yield the desiredresults. The user can also choose their path through the content byclicking on hyperlinks (meta-data) within the mixed-media module. Thisallows the end-user to explore the information that is of the mostinterest to them, in greater detail and in their preferred format (i.e.text, photos, or video). Innovations herein also work across multipleplatforms. For example, mixed-media module interactive components hereincan run inside a standard web browser and its player software can beintegrated into mobile devices, TV devices, video game units, etc.Further, such mixed-media module(s) may be configured as a universalcomponent across all media and devices.

In accordance with aspects of the present innovations, mixed-mediamodules herein can act as an “interactive summary/caption” whichhighlights the curated content from a search result and presents it innarrative form. As such, users may “preview” the contents of the searchin an engaging, interactive experience on multiple devices. In certainimplementations, an interaction a user may have with the mixed-mediamodule is via “Gestures”, such as set forth in connection with FIGS.18-21. These Gestures may include various touch-screen enabledinteractions whereby a user is able to tap, pinch, tap and hold, andswipe or scroll the mixed-media module. Various search engines, serversand/or intermediaries may be configured to respond to or interact inaccordance with these Gestures in different ways, such as the examplesas described in the Figures and associated descriptions herein. Thus,some implementations herein include methods wherein the interactivemultimedia content is configured to allow a new search query andgenerate a new search results page.

FIG. 18 shows an example Gesture consistent with aspects of theinnovations herein. Here, within a search result expanded to theselected mixed-media module, systems and methods herein may beconfigured to respond to a user tap or click of an object in the grid orin the feed to open another mixed-media module, webpage, video, ordetailed animation in an overlay over the current screen. Thus, someembodiments include methods wherein the interaction includes a tap of aportion, button or link of the selected multimedia presentation used inthe generation of the new search results page.

FIG. 19 shows another example Gesture consistent with aspects of theinnovations herein. Here, a user can pinch into an object in the grid tosee detailed or related information on the object including source,related media, access interactive animations, view full video, read fullarticle, and the like. Thus, some embodiments include methods whereininteractions include a pinch of a portion, button or link of theselected multimedia presentation used in the generation of the newsearch results page.

FIG. 20 shows another example Gesture consistent with aspects of theinnovations herein. Here, for example, systems and methods herein may beconfigured such that a user can tap or click and hold on an element inthe grid or in the feed to provide various or additional options. Suchoptions may include, though are not limited to, open now, queue forlater, add to favorites, etc. Thus, some embodiments include methodswherein interactions include a tap and hold of a portion, button or linkof the selected multimedia presentation used in the generation of thenew search results page.

FIG. 21 shows another example Gesture consistent with aspects of theinnovations herein. Here, a user can swipe or scroll with one fingerleft or right over the grid to advance or rewind the presentation of themixed-media. Thus, some embodiments include methods wherein interactionsinclude a swipe or scroll of a portion, button or link of the selectedmultimedia presentation used in the generation of the new search resultspage.

FIG. 22 shows another example of an illustrative interface involving asample search result and mixed-media module presentation, consistentwith aspects of the innovations herein. Here, for example, themixed-media module may be presented as a very foremost piece of content,such as the first item to select in the upper, left portion of thesearch result. Such placement yields easy user access to greater contentin the mixed-media module, and all of the associated benefits therein tothe search engine/provider and ad revenue partners.

In the description here, it is to be understood that both mouse/cursorenabled computing devices, and those without cursors, but use touchscreen technologies are both fully supported. To that, the terms “click”or “tap” or “touch” can be used synonymously and interchangeably. Thus,a clickthrough is the same as a tap-through or any other term with theequivalent meaning. The mobile wireless devices can be touch screenenabled, using a stylus or finger or other such thing to interact withthe screen, and objects on the screen. The touch screen enabledtechnologies also allow for pinching in or out to zoom in or out orenlarge or shrink an object or the display. Sliding a touch can scrolleither in vertical or horizontal directions, or any other directionsupported by the system. The touch screens can also detect a prolongedtap, opening further functionality when a prolonged tap and hold occurson an object. In devices that do not support a touch screen, suchfunctionality can be accomplished by a cursor or pointer of some sort,typically controlled by a mouse, pointer stick, roller ball, etc. Theremay be additional functionality embedded into the display objects toallow for some of the functionality such as a scroll bar or zoombuttons, etc. These functionalities are also fully supported here andcan be used interchangeably with the touch screen enabled technologies.

In the present description, the terms component, module, device, etc.may refer to any type of logical or functional process or blocks thatmay be implemented in a variety of ways. For example, the functions ofvarious blocks can be combined with one another into any other number ofmodules. Each module can be implemented as a software program stored ona tangible memory (e.g., random access memory, read only memory, CD-ROMmemory, hard disk drive) within or associated with the computingelements, sensors, receivers, etc. disclosed above, e.g., to be read bya processing unit to implement the functions of the innovations herein.Or, the modules can comprise programming instructions transmitted to ageneral purpose computer or to processing hardware via a transmissioncarrier wave. Also, the modules can be implemented as hardware logiccircuitry implementing the functions encompassed by the innovationsherein. Finally, the modules can be implemented using special purposeinstructions (SIMD instructions), field programmable logic arrays or anymix thereof which provides the desired level performance and cost.

As disclosed herein, implementations and features of the invention maybe implemented through computer-hardware, software and/or firmware. Forexample, the systems and methods disclosed herein may be embodied invarious forms including, for example, a data processor, such as acomputer that also includes a database, digital electronic circuitry,firmware, software, or in combinations of them. Further, while some ofthe disclosed implementations describe components such as software,systems and methods consistent with the innovations herein may beimplemented with any combination of hardware, software and/or firmware.Moreover, the above-noted features and other aspects and principles ofthe innovations herein may be implemented in various environments. Suchenvironments and related applications may be specially constructed forperforming the various processes and operations according to theinvention or they may include a general-purpose computer or computingplatform selectively activated or reconfigured by code to provide thenecessary functionality. The processes disclosed herein are notinherently related to any particular computer, network, architecture,environment, or other apparatus, and may be implemented by a suitablecombination of hardware, software, and/or firmware. For example, variousgeneral-purpose machines may be used with programs written in accordancewith teachings of the invention, or it may be more convenient toconstruct a specialized apparatus or system to perform the requiredmethods and techniques.

Aspects of the method and system described herein, such as the locationestimate features, may be implemented as functionality programmed intoany of a variety of circuitry, including programmable logic devices(“PLDs”), such as field programmable gate arrays (“FPGAs”), programmablearray logic (“PAL”) devices, electrically programmable logic and memorydevices and standard cell-based devices, as well as application specificintegrated circuits. Some other possibilities for implementing aspectsinclude: memory devices, microcontrollers with memory (such as EEPROM),embedded microprocessors, firmware, software, etc. Furthermore, aspectsmay be embodied in microprocessors having software-based circuitemulation, discrete logic (sequential and combinatorial), customdevices, fuzzy (neural) logic, quantum devices, and hybrids of any ofthe above device types. The underlying device technologies may beprovided in a variety of component types, e.g., metal-oxidesemiconductor field-effect transistor (“MOSFET”) technologies likecomplementary metal-oxide semiconductor (“CMOS”), bipolar technologieslike emitter-coupled logic (“ECL”), polymer technologies (e.g.,silicon-conjugated polymer and metal-conjugated polymer-metalstructures), mixed analog and digital, and so on.

It should also be noted that the various logic and/or functionsdisclosed herein may be enabled using any number of combinations ofhardware, firmware, and/or as data and/or instructions embodied invarious machine-readable or computer-readable media, in terms of theirbehavioral, register transfer, logic component, and/or othercharacteristics. Computer-readable media in which such formatted dataand/or instructions may be embodied to include, but are not limited to,non-volatile storage media in various forms (e.g., optical, magnetic orsemiconductor storage media), though do not include non-tangible media.

Unless the context clearly requires otherwise, throughout thedescription and the claims, the words “comprise,” “comprising,” and thelike are to be construed in an inclusive sense as opposed to anexclusive or exhaustive sense; that is to say, in a sense of “including,but not limited to.” Words using the singular or plural number alsoinclude the plural or singular number respectively. Additionally, thewords “herein,” “hereunder,” “above,” “below,” and words of similarimport refer to this application as a whole and not to any particularportions of this application. When the word “or” is used in reference toa list of two or more items, that word covers all of the followinginterpretations of the word: any of the items in the list, all of theitems in the list and any combination of the items in the list.

Other implementations of the invention will be apparent to those skilledin the art from consideration of the specification and practice of theinvention disclosed herein. It is intended that the specification andexamples be considered as exemplary only, with a true scope and spiritof the invention being indicated by the disclosure above in combinationwith the following paragraphs describing the scope of one or moreimplementations of the following invention.

The invention claimed is:
 1. A method comprising: receiving, by acomputing device, a search query input by a user; identifying, by thecomputing device, real-time data sources based on the search query anduser preferences for the real-time data sources, the real-time datasources comprising third-party websites frequently accessed by a userand serving unstructured data via webpages; retrieving, by the computingdevice, real-time updated information related to the search query fromthe real-time data sources, the real-time updated information comprisingrecently updated information published on the webpages of the real-timedata sources; parsing, by the computing device, the real-time updatedinformation into metadata, the parsing comprising: extracting aplurality of content items from the webpages based on informationselected from the group consisting of a user profile, user searchhistory, user preferences, and user customization, a content itemcomprising one or more of image, video, or audio content present withinthe real-time information, deduplicating the plurality of content itemsto obtain a deduplicated set of content items, and assigning thededuplicated set of content items as the metadata; ranking, by thecomputing device, the metadata based on information selected from thegroup consisting of the user profile, the user search history, the userpreferences, and the user customization; and generating, by thecomputing device, an interactive mixed-media module based on rankedmetadata, the interactive mixed-media module comprising a controllablemedia component displaying a first portion of the plurality of contentitems and a navigational control for navigating to a second portion ofthe plurality of content items.
 2. The method of claim 1, wherein themetadata comprises biographical information.
 3. The method of claim 1,further comprising publishing, by the computing device, the interactivemixed-media module on a search engine results page.
 4. The method ofclaim 1, wherein the search query is a news event.
 5. The method ofclaim 1, wherein the ranking of the metadata further comprises rankingthe metadata based on information selected from information typesconsisting of popularity, information associated with an advertiser, anda search engine index.
 6. The method of claim 1, wherein the interactivemixed-media module comprises an embedded executable link related to themetadata.
 7. The method of claim 1, wherein the receiving of the searchquery further comprises receiving an incomplete input character stringfrom the user.
 8. A computing device comprising: a processor; a storagemedium for tangibly storing thereon program logic for execution by theprocessor, the program logic comprising: receiving logic executed by theprocessor for receiving a search query input by a user; identificationlogic executed by the processor for identifying real-time data sourcesbased on the search query and user preferences for the real-time datasources, the real-time data sources comprising third-party websitesfrequently accessed by a user and serving unstructured data viawebpages; retrieving logic executed by the processor for retrievingreal-time updated information related to the search query from thereal-time data sources, the real-time updated information comprisingrecently updated information published on the webpages of the real-timedata sources; parsing logic executed by the processor for parsing, thereal-time updated information into metadata, the parsing logiccomprising: extraction logic executed by the processor for extracting aplurality of content items from the webpages based on informationselected from the group consisting of a user profile, user searchhistory, user preferences, and user customization, a content itemcomprising one or more of text, image, video, or audio content presentwithin the real-time information, deduplication logic executed by theprocessor for deduplicating the plurality of content items to obtain adeduplicated set of content items, and assignment logic executed by theprocessor for assigning the deduplicated set of content items as themetadata; ranking logic executed by the processor for ranking themetadata based on information selected from the group consisting of theuser profile, the user search history, the user preferences, and theuser customization; and generating logic executed by the processor forgenerating an interactive mixed-media module based on ranked metadata,the interactive mixed-media module comprising a controllable mediacomponent displaying a first portion of the plurality of content itemsand a navigational control for navigating to a second portion of theplurality of content items.
 9. The computing device of claim 8, furthercomprising publishing logic executed by the processor for publishing theinteractive mixed-media module on a search engine results page.
 10. Thecomputing device of claim 8, wherein the interactive mixed-media modulecomprises an embedded executable link related to the metadata.
 11. Anon-transitory computer readable storage medium for tangibly storingcomputer program instructions capable of being executed by a computerprocessor of a computing device, the computer program instructionsdefining the steps of: receiving, by the computer processor, a searchquery input by a user; identifying, by the computer processor, real-timedata sources based on the search query and user preferences for thereal-time data sources, the real-time data sources comprisingthird-party websites frequently accessed by a user and servingunstructured data via webpages; retrieving, by the computer processor,real-time updated information related to the search query from thereal-time data sources, the real-time updated information comprisingrecently updated information published on the webpages of the real-timedata sources; parsing, by the computer processor, the real-time updatedinformation into metadata, the parsing comprising: extracting aplurality of content items from the webpages based on informationselected from the group consisting of a user profile, user searchhistory, user preferences, and user customization, a content itemcomprising one or more of text, image, video, or audio content presentwithin the real-time information, deduplicating the plurality of contentitems to obtain a deduplicated set of content items, and assigning thededuplicated set of content items as the metadata; ranking, by thecomputer processor, the metadata based on information selected from thegroup consisting of the user profile, the user search history, the userpreferences, and the user customization; and generating, by the computerprocessor, an interactive mixed-media module based on ranked metadata,the interactive mixed-media module comprising a controllable mediacomponent displaying a first portion of the plurality of content itemsand a navigational control for navigating to a second portion of theplurality of content items.